Skip to content

feat: agent observatory schema foundation + transcript-forwarding env rename - #173

Open
jmagar wants to merge 23 commits into
mainfrom
feat/agent-observatory-env-rename-20260801
Open

feat: agent observatory schema foundation + transcript-forwarding env rename#173
jmagar wants to merge 23 commits into
mainfrom
feat/agent-observatory-env-rename-20260801

Conversation

@jmagar

@jmagar jmagar commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Lays the Agent Observatory schema foundation: repository, worktree, git-commit, repository-observations, agent-run, and run-attribution schemas, plus locked contract versions (AGENT_OBSERVATORY_SCHEMA_VERSION / AGENT_OBSERVATORY_PROJECTION_VERSION) with a compile-time check that the runtime schema (db::KNOWN_SCHEMA_VERSION) never claims to have already applied migrations that don't exist yet. This check lives in the #[cfg(test)] module, so it is evaluated whenever the test target is compiled (cargo test, cargo build --tests) — it is not evaluated by a plain cargo build/cargo build --release.
  • Renames the transcript-forwarding environment switch to a clearer, normalized config key, with doctor detection and safe automatic migration for the old key (ENV-003), a strict occurrence allowlist plus Cortex 4.0 removal documentation for the legacy name (ENV-004). Migration rewrites only the single detected legacy line (not a whole-file substring replace), so incidental occurrences of the key elsewhere in the .env file (e.g. a stale comment) are left untouched.
  • Adds docs/contracts/agent-observatory.md, docs/design/agent-observatory-architecture.md, docs/design/agent-observatory-ui.md, docs/research/2026-07-31-agent-observatory.md, and docs/specs/agent-observatory.md, and wires the AO-001/ENV-004 verification scripts into the Justfile (check-agent-observatory-contracts, validate-transcript-forward-env-rename).
  • Resolves clippy::manual_strip / clippy::collapsible_if lints introduced in src/setup/doctor.rs.
  • Merges current main to pick up the already-merged toolchain-drift cleanup (chore(build): remove hidden binary artifact wrapper #167) rather than duplicating it.

Test plan

  • cargo check --locked clean after merging main
  • cargo clippy --all-targets --all-features --locked -- -D warnings clean
  • Focused regression test for the line-anchored .env migration rewrite
  • just check-agent-observatory-contracts and just validate-transcript-forward-env-rename both exit 0
  • Pre-commit hooks (rustfmt, diff/module-size/env-guard checks) passed

jmagar and others added 23 commits August 1, 2026 02:06
…n for transcript forwarding env rename

Add doctor migration check for deprecated CORTEX_AGENT_AI_TRANSCRIPTS →
CORTEX_AGENT_AI_TRANSCRIPT_FORWARD rename with safe atomic rewrites under
--fix --yes authorization.

Implementation:
- check_transcript_forward_env_migration() in doctor.rs detects four cases:
  1. Neither set: OK, no action
  2. New only: OK, already correct
  3. Legacy only: Warn without --fix, rename key atomically with --fix --yes
  4. Both equal: Warn without --fix, remove legacy line with --fix --yes
  5. Conflicting: Error, no write (manual resolution required)
- migrate_legacy_only(): atomically rename key preserving value and permissions
- migrate_both_equal(): atomically remove legacy line preserving permissions
- atomic_write_env_file(): POSIX O_NOFOLLOW atomic write matching firstrun.rs
  - Tempfile with pid.nanos suffix
  - 0o600 permissions
  - sync_all before rename
  - fsync parent directory

Test coverage (15 tests in doctor_tests.rs):
- Detection: legacy_only, both_equal, conflicting, new_only, neither, missing
- Authorization: fix without --yes warns, --fix --yes executes
- Atomic behavior: rename preserves value and 0o600 permissions
- Idempotent: second run is no-op
- Conflict safety: error leaves file unchanged

Gate: no migration occurs without explicit --fix --yes authorization.
… documentation

Add validation infrastructure to ensure deprecated CORTEX_AGENT_AI_TRANSCRIPTS
only appears in approved compatibility locations, plus Cortex 4.0 removal checklist.

Validation script (scripts/validate-transcript-forward-env-rename.sh):
- Strict occurrence allowlist: legacy variable only permitted in:
  - Compatibility resolver/tests (heartbeat_agent.rs, heartbeat_agent_tests.rs)
  - Doctor migration (doctor.rs, doctor_tests.rs)
  - Deployment regression tests (agent_deploy_tests.rs)
  - Setup generation (heartbeat_agent.rs in src/setup/)
  - Plan/spec/docs (01a plan, contract, research, specs)
  - CHANGELOG.md (deprecation entry)
  - This validation script
- Verifies new variable is properly documented
- Checks .env.example uses only new name
- Reports violations with file:count evidence

Contract documentation (docs/contracts/agent-observatory.md):
- Section 10.2: Cortex 4.0 removal checklist
- Complete removal to-do list for coordinated breaking change:
  - Parser compatibility code removal
  - Doctor migration function removal
  - Compatibility test removal
  - Documentation updates
  - Validation script removal
  - Setup generation cleanup
- Verification steps to confirm complete removal

Gate: removal before Cortex 4.0 fails compatibility test; all artifacts
(parser, warnings, migration, tests, docs, allowlist) deleted together.
origin/main removed the hidden binary-artifact wrapper in #167
(846be08), relying solely on the global rustc-wrapper (kache) from
~/.cargo/config.toml. This branch forked before that removal and
still carried the local scripts/cargo-rustc-wrapper override, which
shells out to `sccache`, now retired on this host with no mise
version selected. That broke every build, including the pre-push
hook's `cargo xtask pre-push`.

Bring .cargo/config.toml, Justfile, and scripts/ back in line with
main: drop the local rustc-wrapper override and the wrapper test
script. Builds now go through the fleet-standard kache cache.
clippy::assertions_on_constants flagged the runtime assert! comparing
AGENT_OBSERVATORY_SCHEMA_VERSION against db::KNOWN_SCHEMA_VERSION under
--all-targets --all-features -D warnings. Both are const i64, so the
ordering invariant is provable at compile time via `const _: () =
assert!(...)` instead of a #[test] runtime check.
migrate_legacy_only() previously used a whole-file str::replace on
CORTEX_AGENT_AI_TRANSCRIPTS=, which would also mangle unrelated
occurrences of that substring elsewhere in the .env file (e.g. a
comment left behind from a prior manual migration). Rewrite only the
single legacy_line identified by the same line-anchored trim() +
strip_prefix(KEY) + strip_prefix('=') match the detection logic above
already uses, mirroring migrate_both_equal's existing pattern.

Adds a regression test asserting a comment containing the legacy key
as a substring survives --fix --yes untouched while the real
assignment is renamed.
The five new Agent Observatory planning docs (contract, architecture,
UI design, research ledger, spec) were missing the title/created/
updated frontmatter block the Repository Contract check requires,
matching the format already used across docs/ (e.g.
docs/adr/001-sqlite-single-writer.md, docs/CLI.md).
scripts/check-agent-observatory-contracts.sh (AO-001's own declared
deliverable) and scripts/validate-transcript-forward-env-rename.sh
(ENV-004) existed but had no Justfile target, so neither was runnable
via `just` or discoverable through `just --list`. Add
check-agent-observatory-contracts and
validate-transcript-forward-env-rename recipes, matching AO-001's
proof text ("just check-agent-observatory-contracts exits 0").
pool_tests.rs used the literal real dev-host name "dookie" (and
"squirts") as sample fixture data in ~15+ places. Swap to the
synthetic devhost/edgehost names already adopted by the repo's
internal-identifier scrub for this category of fixture.
The doc comment on the const _ schema-ordering assertion said it
"catches drift at build time instead of only when cargo test happens
to run", which reads as though a plain cargo build/cargo build
--release is protected. It isn't: the assertion lives in a
#[cfg(test)] module, so it only evaluates when the test target is
compiled (cargo test, cargo build --tests). Tighten the comment to say
so explicitly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant